Papers with event extraction research
Evaluating Zero-Shot Event Structures: Recommendations for Automatic Content Extraction (ACE) Annotations (2023.acl-short)
Copied to clipboard
| Challenge: | Zero-shot event extraction (EE) methods infer richly structured event records from unstructured text data, based on a user-supplied natural language specification and no training examples. |
| Approach: | They propose recommendations for future evaluations so the research community can better utilize ACE as an event evaluation resource. |
| Outcome: | The proposed methods can be used to evaluate zero-shot and other low-supervision EE methods, considering up to 32% of correctly identified arguments and 25% of correctly ignored event mentions as false negatives. |
GLEN: General-Purpose Event Detection for Thousands of Types (2023.emnlp-main)
Copied to clipboard
| Challenge: | ACE 2005 2 is the first large-scale event extraction dataset with 205K event mentions and 3,465 different types. |
| Approach: | They propose to use the DWD Overlay to map PropBank rolesets to a large distantlysupervised training dataset with partial labels to make event extraction more accessible. |
| Outcome: | The proposed model performs better than baselines including InstructGPT and ACE 2005 2 despite being 18 years old . key limitations of ACE include its small event ontology of 33 types, small dataset size of around 600 documents and restricted domain (with a significant portion concentrated on military conflicts). |
EXCEEDS: Extracting Complex Events via Nugget-based Grid Modeling in Scientific Domain (2026.acl-long)
Copied to clipboard
| Challenge: | Extensive event extraction research has been conducted in many domains, including news, finance, and biology. |
| Approach: | They propose an end-to-end scientific event extraction framework for encoding nuggets into a grid matrix and simplifying complex event extraction as a nuggot-based grid modeling task. |
| Outcome: | The proposed framework performs well in scientific domain, demonstrating state-of-the-art performance. |
TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction (2024.findings-acl)
Copied to clipboard
Kuan-Hao Huang, I-Hung Hsu, Tanmay Parekh, Zhiyu Xie, Zixuan Zhang, Prem Natarajan, Kai-Wei Chang, Nanyun Peng, Heng Ji
| Challenge: | Recent studies suggest that event extraction evaluations may not accurately reflect the true performance. |
| Approach: | They propose a standardized, fair, and reproducible benchmark for event extraction . they use standardized scripts and splits for 16 datasets spanning eight domains . |
| Outcome: | The proposed benchmarks show that they struggle to achieve satisfactory performance. |